Time Series and Data Management ======================================= Dates ------- In the RISE toolbox, dates play a crucial role in representing time series data. Dates can be either explicit, where each observation is associated with a specific date, or undated, where dates are simply represented as a sequence of integers. To handle dates efficiently, RISE provides an abstract class called ``rise_dates.dates``, from which various derived classes inherit. These derived classes include: - ``rise_dates.DailyDate`` (shortcut: ``rd``) - ``rise_dates.WeeklyDate`` (shortcut: ``rw``) - ``rise_dates.MonthlyDate`` (shortcut: ``rm``) - ``rise_dates.QuarterlyDate`` (shortcut: ``rq``) - ``rise_dates.HalfYearlyDate`` (shortcut: ``rh``) - ``rise_dates.YearlyDate`` (shortcuts: ``ry`` or ``ra``) Each derived class represents a specific frequency of dates, such as daily, weekly, monthly, etc. Base Class Methods ~~~~~~~~~~~~~~~~~~~~ The base class ``rise_dates.dates`` provides several methods that can be used with any derived class. These methods allow you to manipulate and analyze date objects effectively. Here are the commonly used methods: - ``char()``: Converts dates to character representations. - ``convert()``: Converts dates to another frequency. - ``datestr()``: Returns a string representation of dates. - ``eq()``: Checks if dates are equal. - ``gt()``: Checks if dates are greater than a given date. - ``ismember()``: Checks if dates are members of a given set of dates. - ``lt()``: Checks if dates are less than a given date. - ``min()``: Returns the minimum date. - ``ne()``: Checks if dates are not equal. - ``unique()``: Returns unique dates in a sequence. - ``colon()``: Creates a sequence of dates. - ``double()``: Converts dates to double format. - ``ge()``: Checks if dates are greater than or equal to a given date. - ``intersect()``: Returns the intersection of two sets of dates. - ``le()``: Checks if dates are less than or equal to a given date. - ``max()``: Returns the maximum date. - ``minus()``: Computes the difference between two dates. - ``plus()``: Adds a given number of periods to a date. - ``ymd()``: Extracts the year, month, and day components of dates. These methods provide powerful functionality for manipulating, comparing, and performing operations on date objects. For more detailed information on each method, refer to the documentation. Help Text for Derived Classes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To make it easier for users to understand and use the derived classes, we have provided detailed help text for each of them (``rd``, ``rw``, ``rm``, ``rq``, ``rh``, ``ry``, and ``ra``). The help text contains information about the specific properties and methods available for each class. .. index:: rd .. _rd: Daily dates : rd """""""""""""""""" .. include:: rd_help.rst .. index:: rw .. _rw: Weekly dates : rw """""""""""""""""" .. include:: rw_help.rst .. index:: rm .. _rm: Monthly dates : rm """"""""""""""""""" .. include:: rm_help.rst .. index:: rq .. _rq: Quarterly dates : rq """"""""""""""""""""" .. include:: rq_help.rst .. index:: rh .. _rh: Half-yearly dates : rh """"""""""""""""""""""" .. include:: rh_help.rst .. index:: ry .. _ry: Yearly dates : ry """""""""""""""""" .. include:: ry_help.rst .. index:: ra .. _ra: Annual dates : ra """""""""""""""""" .. include:: ra_help.rst Once you are familiar with the concepts and usage of date objects in the RISE toolbox, you can proceed to explore the handling of time series data, which builds upon the foundation of dates. .. _dates-technicals: Technical documentation for rise_dates.dates objects ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. toctree:: :maxdepth: 2 :caption: Contents: rise_dates_properties_methods Time Series ------------- This section provides information about time series and their usage in the context of the RISE toolbox. It covers topics such as: - Introduction to time series analysis - Working with time series data in RISE - Time series transformations and manipulations - Statistical analysis techniques for time series Time series data in essence is just a matrix, so in theory, one can just use matrices to represent the data. However, all practitioners will share their frustration in trying to remember the index location of the GDP and the annoyance in trying to find out what ``data(:,5)`` written by a co-worker represents. The RISE toolbox addresses this problem by providing a time series object that uses natural indices, improving readability and reducing errors. To use the time series object, you can initialize it with the following code: .. code-block:: matlab db = ts('1990q1', randn(10,4), {'gdp', 'r', 'wage', 'capital'}) The first argument specifies the starting date of the time series and its frequency. In this example, the time series starts from the first quarter of 1990. The second argument is the data matrix, where each row represents a time period and each column represents a variable. Here, the time series consists of 4 variables spanning 10 quarters. Lastly, the third argument provides the natural names of the variables for intuitive indexing. If you want to provide descriptions or comments for each variable, you can construct the time series object with optional descriptions: .. code-block:: matlab db = ts('1990q1', randn(10,2), {'gdp', 'pi'}, {'real gdp', 'inflation rate'}) You can display the time series, including its descriptions/comments, by calling: .. code-block:: matlab disp(db) Alternatively, you can set variable names and descriptions using the ``set`` function: .. code-block:: matlab set(db, 'varnames', {'gdp', 'pi'}, 'description', {'real gdp', 'inflation rate'}) .. _ts-indexing: Indexing and assignment ~~~~~~~~~~~~~~~~~~~~~~~~ The indexing surface of the time series object mirrors the built-in ``table`` / ``timetable`` conventions, so that users familiar with those classes feel at home, and the distinction between "sub-time-series" and "raw values" is explicit. ================================ ========================================================= Expression Result ================================ ========================================================= ``db.PROP`` a property of the ts (``data``, ``dates``, ``frequency``, ``NumberOfObservations``, ...). ``db.VARNAME`` if ``VARNAME`` matches a variable name (and is not a property/method), returns the single-variable ts. ``db(I[,J[,K]])`` a **sub-time-series** (returns a ts). ``I``, ``J``, ``K`` are dates/date strings/logical/cellstr/positions/``:``/ function-handle filters on pages. ``db{I[,J[,K]]}`` the **raw numeric block** at that selection (a double array of size ``nobs x nvars x npages``). Use this when the next step is plain MATLAB arithmetic; otherwise stay in the ts world with ``()``. ``db{-k}`` / ``db(-k)`` (**dated ts only**, ``k`` a scalar integer) shortcut for ``lag(db, k)``; returns a ts. ``db{+k}`` / ``db(+k)`` is ``lead(db, k)``. ``db{0}`` is ``db``. On undated ts a scalar numeric is the implicit date value instead. ``db.VARNAME = rhs`` replace one variable's column from a ts or a numeric column. ``db(I,...) = rhs`` replace a sub-time-series (rhs ts or numeric). ``db{I,...} = rhs`` raw numeric block write. ================================ ========================================================= Scalar-numeric assignment (``db{-1} = X``, ``db(-1) = X``) is **not** supported. The time-shift shortcut is read-only; build the shifted ts first and assign with a date or logical selector if you need to write into it. Examples: .. code-block:: matlab db = ts('1990q1', randn(10,4), {'gdp', 'r', 'wage', 'capital'}); % Access values db.wage % single-variable ts db('wage') % same, sub-ts form db{'wage'} % raw 10x1 double array % Slice rows and columns db('1991q2', 'wage') % single observation, as a ts db{'1991q2', 'wage'} % the same value as a 1x1 double db('1990q2:1991q2', {'gdp','wage'}) % sub-ts (5 rows, 2 vars) db{:, {'gdp','wage'}} % full sample, two vars, as a 10x2 double % Whole-year selection works across frequencies db(ry(1991)) % the 4 quarters of 1991, as a ts % Underlying data, when you need plain MATLAB values(db) % equivalent to db.data Time shifts (lags and leads) are expressed either with the explicit methods ``lag(db, k)``, ``lead(db, k)``, ``shift(db, k)`` or, on a dated ts, with the brace/paren shortcut. Both preserve the date axis - the leading or trailing ``k`` observations are NaN-padded so that the length of the output equals the length of the input. The two forms are equivalent on a dated ts: .. code-block:: matlab growth = 100*(db.gdp / lag(db.gdp, 1) - 1); % q/q growth, % terms growth = 100*(db.gdp / db.gdp{-1} - 1); % same thing, shorter On an undated ts the shortcut is unavailable (a scalar numeric in the braces is treated as the date value itself); use ``lag``/``lead``/ ``shift`` explicitly there. The time series object seamlessly integrates with other parts of the RISE toolbox, enabling you to perform VAR and DSGE analyses effortlessly. The automation provided by the time series object simplifies various data management tasks. The following are some commonly used functions and their functionalities: Data Cleaning ~~~~~~~~~~~~~~~~~ - `reset_data`: Resets the data in the time series object. - `reset_start_date`: Resets the start date of the time series object. - `interpolate`: Fills in missing data by interpolating. - `aggregate`: Aggregates data to a coarser frequency (e.g., monthly data to quarterly). - `dummy`: Creates a dummy variable. - `step_dummy`: Creates a step dummy variable. - `cat`: Joins multiple time series objects. Time Series Transformations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - ``hpfilter``: Applies the Hodrick-Prescott (HP) filter to the time series data. - ``ma_filter``: Applies a moving-average filter to the time series data. - ``pdecomp``: Performs a parametric trend-seasonal-residual decomposition. - ``npdecomp``: Performs a non-parametric trend-seasonal-residual decomposition. - ``transform``: applies a standard time-series transformation (level, growth, difference, log change, year-over-year variants) following the Haver mnemonics. Time shifts ~~~~~~~~~~~~ These methods preserve the date axis - the leading or trailing ``k`` rows are NaN-padded so that ``length(out) == length(db)``: - ``lag(db, k)``: lag-by-k. ``out.data(t,:,:) = db.data(t-k,:,:)``. - ``lead(db, k)``: lead-by-k. ``out.data(t,:,:) = db.data(t+k,:,:)``. - ``shift(db, k)``: generic shift. Positive ``k`` is a lead, negative is a lag. On a dated ts these are also available as the shortcut ``db{-k}`` / ``db(-k)`` (lag) and ``db{+k}`` / ``db(+k)`` (lead) - see the :ref:`ts-indexing` section above. Statistical Analysis ~~~~~~~~~~~~~~~~~~~~~~~ - ``regress``: Performs OLS regression on the time series data. - ``chowlin``: Temporal disaggregation using the Chow-Lin method (interpolates a low-frequency series to high frequency using one or more related indicators). - ``jbtest``: Performs the Jarque-Bera (JB) test. Plotting tools for time series ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - `bar`: Creates a bar plot. - `fanchart`: Generates a fan chart. - `plot`: Plots the time series data. - `plot_real_time`: Plots the time series data in real time conditioning. - `plotyy`: Creates a plot with two separate axes. Other Automations ~~~~~~~~~~~~~~~~~~~~ - `transform`: Provides a variety of transformations for the time series data. For a comprehensive understanding of all the automation and functionalities available in the time series object, refer to the `ts-technicals` documentation. .. _ts-technicals: Technical documentation for ts objects ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. toctree:: :maxdepth: 2 :caption: Contents: ts_properties_methods